Screen watermarking methods and arrangements

ABSTRACT

In an exemplary embodiment, image content is rendered to include a digital watermark pattern at an essentially fixed scale, regardless of the size of screen on which the content is displayed. In one particular embodiment, CSS and JavaScript are responsive to the rendering context (which can include screen size, and/or viewer position) to define the geometry of a watermarked texture block, which is tiled to serve as a background of an HTML-defined page. A great variety of other features and arrangements are also detailed.

RELATED APPLICATION DATA

This application claims priority to provisional applications 61/970,233,filed Mar. 25, 2014, and 62/065,462, filed Oct. 17, 2014.

INTRODUCTION

Digital watermark technology can facilitate in-person sharing of digitalcontent.

Without digital watermarking, if a first person wants to find and obtaina copy of content displayed on a second person's desktop computermonitor or smartphone display (e.g., a game, a video, or a blog post),there is no straightforward way to do so.

The first person might verbally ask the second person for a URL, andthen write it down to access later. But that's slow and error-prone.

Or the first person could ask the second person to send a URL (or thecontent itself) to an account of the first person, such as by text,email, a social networking account (e.g., Reddit, Digg, Pinterest,etc.), or by a proprietary data exchange technology (e.g., Apple'sAirDrop or iCloud, or Google Drive). But that can also be tedious.Moreover, it involves interrupting the second person's activity andlaunching an associated communication tool on the second person'sdevice.

Digital watermarking can be employed to enable the first person simplyto snap an image of the second person's display screen with asmartphone, and obtain a copy of the content by reference to thecaptured imagery. (Software in the smartphone processes the capturedimagery to decode an identifier of the displayed content (e.g., a URL),and then uses the smartphone browser to load the identified content.)The first person obtains a copy quickly, without disrupting the secondperson.

While such functionality has been possible in a technical sense for manyyears (e.g., as in patent publication 20100226526), various impedimentshave made it difficult as a practical matter.

One issue has been the task of encoding the content with the necessarydigital watermark information. Each content author needed to take theextra step of adding a watermark to the content.

In accordance with one aspect of the present technology, the task ofwatermark encoding is not left to the content author. Instead, awatermark (e.g., specifying a URL for the content) is encoded in thedisplayed content without involvement of the content author.

Another issue has been one of physical scale, due to thedifferently-sized screens on which content might be displayed.

Imagery captured from a five inch smartphone display will represent anembedded digital watermark pattern at a much smaller scale than the sameimagery captured from a 21 inch desktop monitor, posing difficulties towatermark decoding. (Related issues arise when a user re-sizes a displaywindow, or uses a control to zoom-in or -out on displayed content.)

In accordance with another aspect of the technology, the physical scaleof a digital watermark in electronic imagery is de-coupled from thephysical size of the digital display (or the size of the display window,or the zoom level).

Watermarking is most commonly implemented in the content-creationprocess. U.S. Pat. No. 6,899,475, however, details an alternateapproach—in which watermark encoding is performed in the renderingprocess (i.e., the process of outputting content to viewable form). Inthe patented system, a printer executes an interpreter that rasterizesPostScript-defined imagery for printing on paper. The interpreter alsogenerates a pattern corresponding to a desired payload, computes a maskidentifying excerpts of the imagery that can be modified without visualartifacts, and modifies the rasterized output to steganographicallyencode the pattern—processed with the mask—in the printed output. Sucharrangement, however, is computationally intense.

Another alternate approach is used in digital cinema. The arrangement issimilar to the Postscript printer example, except that the masking datais computed as part of the content creation process, and is thenco-conveyed with the MPEG data stream. At the time of rendering, aprocessor generates a pattern corresponding to a desired payload (e.g.,indicating the date, time and place of rendering), weights this patternwith the masking data, and combines it with the MPEG image data forrendering on a projection screen. This arrangement, however, burdens thecontent creation task.

In accordance with a further aspect of the present technology,watermarking is performed during rendering, but without the complexityof the foregoing techniques.

Cascading Style Sheets (CSS) is a style sheet language familiar toartisans, used for describing the look and formatting of a documentwritten in a markup language. While most often used to style web pagesand interfaces written in HTML and XHTML, the language can be applied toany kind of XML document. CSS serves primarily to enable the separationof document content from document presentation, including elements suchas the layout, colors, and fonts.

In accordance with still another aspect of the present technology, CSSis employed to include a watermark pattern as part of a document'spresentation at the time of rendering. The CSS instructions can definenot only the size of the presented watermark pattern (e.g., the size ofthe component watermark tile), but can also specify the desiredwatermark tile by payload, or other parameter. This allows documents tobe dynamically marked with different watermark patterns (e.g.,triggering different actions), based on various factors, such ascontext.

The foregoing and a great variety of other features and arrangementswill be more readily apparent from the following Detailed Description,which proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A, 1B and 1C show an image, watermarked using prior art methods,displayed on three different display devices.

FIGS. 2A, 2B and 2C show the differing scales of component watermarkblocks in FIGS. 1A, 1B, and 1C, respectively.

FIGS. 3A, 3B and 3C illustrate an aspect of certain embodiments of thepresent technology: consistent image watermark block size acrossdifferent display devices.

FIG. 4 shows that the scale of a component watermark block issubstantially consistent across all of FIGS. 3A, 3B and 3C.

FIGS. 5A and 5B illustrate that a translucent watermark image can beadded atop other imagery (or that a flat watermark image and otherimagery can be blended).

FIGS. 6A and 6B show a web page displayed at zoom levels of 75% and110%.

FIG. 7A shows an example of CSS instructions used with one aspect of thepresent technology.

FIG. 7B shows an example of JavaScript instructions used with one aspectof the present technology.

FIG. 7C shows an example of HTML instructions used with one aspect ofthe present technology

FIGS. 8A, 8B, and 8C show text in a display window, with zoom levels of100%, 90%, and 110%.

FIG. 9 shows CSS instructions for use with the Firefox browser.

FIG. 10 illustrates an exemplary lookup table data structure, by whichcaptured image content can be associated with online content.

FIG. 11 illustrates how a uniform pattern of dots appears distorted,based on the shape of a surface onto which it is projected.

DETAILED DESCRIPTION

Digital watermark technology is widely used in electronic and printmedia, e.g., to enable a smartphone to link to associated content. Forexample, a page in a magazine article may be digitally watermarked witha URL of a web site that has further information related to the subjectof the article. A user captures an image of the printed page with thesmartphone camera, and software extracts the URL and links to thecorresponding web page. Such arrangements are detailed, e.g., in U.S.Pat. Nos. 6,947,571 and 8,620,021.

In print media, a digital watermark signal typically takes the form of arectangular block pattern, about an inch on a side, which isrepetitively tiled across the printed content. The pattern serves tovary the luminance or chrominance of the underlying print content to avery small degree—too small to be noticeable to a human viewer undernormal conditions, but detectable by computer analysis ofcamera-captured imagery. Each block pattern encodes a data payload ofabout 50 or 100 binary bits. The tiling of many such identical blockpatterns across the print media allows the payload to be recovered fromcamera imagery captured from any location on the page; e.g., an 8×10″printed page may have 80 such watermark blocks across its extent.(Exemplary watermark technology is detailed in U.S. Pat. Nos. 6,590,996and 8,199,969.)

Watermark detection software, such as on a smartphone, expects thewatermark in captured imagery to be of a certain physical scale (e.g.,corresponding to a camera-to-page viewing distance of six inches).However, the detector is robust to a range of scale states, e.g.,varying between about 50% and 200% of the nominally-expected values.Thus, a print watermark may be read from a distance not just of sixinches, but anywhere in a range of three to twelve inches.

The scale of a print watermark is fixed at the time of printing. Thephysical page size does not later change.

The situation is different for watermarks found in electronic imagery,such as at online web sites and videos. Such electronic imagery may bedisplayed on a 5 inch smartphone screen, or on a 21 inch desk monitor.Moreover, the user may cause the size of the rendered content to vary,such as by changing the size of a window in which the electronic contentis presented, or by zooming-in/zooming-out. These actions can causeextreme variations in the physical scale at which the electronic imageryis displayed, and subsequently captured.

Like print imagery, electronic imagery is typically watermarked usingtiled pattern blocks. But the size of each block can vary over a verylarge range, depending on the just-noted display conditions. Whiledigital watermark detection software is robust across certain scaleranges, it is not suited for the 10:1, or more, variations in scale thatmay occur with display of electronic content.

This phenomenon is depicted in FIGS. 1A-1C, which show the familiarimage of the American flag being raised on Iwo Jima. The image isencoded with a digital watermark, in conventional fashion (illustratedas dashed-line squares across the image, but essentially imperceptibleto humans). When the image is shown on a smartphone display (FIG. 1A),it appears at a diagonal scale of about five inches. When the same imageis shown on a tablet or desktop display (FIGS. 1B, 1C), it appears at adiagonal scale of about 12 or 21 inches, respectively.

The watermark pattern similarly scales with the image into which it isembedded. Thus, the width of a component watermark tile is about ahalf-inch in FIG. 1A, versus about 1.2 inches in FIG. 1B, and 2.3 inchesin FIG. 1C. These different watermark scales are shown in FIGS. 1A-1C.

A similar scaling effect may occur if a window presenting the image, inthe desktop display screen of FIG. 1C, is resized (e.g., by clicking acorner of the window and dragging, as is familiar from graphical userinterfaces). Similarly, the rendering software may allow the user tozoom-in or zoom-out on the image, again changing the scale of theembedded watermark pattern.

FIGS. 3A-3C show the same image and watermark, in accordance with anaspect of certain embodiments of the present technology. In the depictedarrangement, the watermark is not encoded into the image data. Rather,it is separate. For example, the rendering software that controlspresentation of information on the screen may present the watermarkpattern as a translucent overlay, together with presentation of theunderlying Iwo Jima image. (Blending of the Iwo Jima image with awatermark image of the same size, in a weighted arrangement—such as byweighted averaging—is another way this operation can be performed.)

Regardless of the size of screen on which the imagery is presented(e.g., smartphone, tablet, or desktop computer), the block size of thewatermark pattern is maintained substantially constant. FIG. 4 shows thesize of the resulting watermark block; it is essentially unchanged,irrespective of the display size. (“Substantially,” as used herein,means within 20% of its original value. “Essentially” means within 10%.)

In the illustrated example, the resulting watermark block is one inchacross. (In typical implementations, the watermark block may range froma half inch to two inches in width—or from about 5% to 20% of thelongest physical dimension of the display screen, although valuesoutside this range can also be used.) This block size is also maintainedsubstantially constant despite re-sizing of display windows, andzooming.

Such arrangement is further illustrated in FIGS. 5A and 5B. In both, animage of a cat is blended, in weighted arrangement, with asimilarly-sized frame of digital watermark pattern. In this case thewatermark pattern appears—on human viewing—to be a flat, mid-grey.(There are actually subtle variations that escape human notice in theluminance or chrominance of the grey pixels across each of the componentwatermark blocks; it is these variations that encode the watermarkpayload.)

In FIG. 5A, there is a 90/10 weighting between the cat and the greywatermark images. In FIG. 5B, there is a 75/25 weighting between the catand watermark images. (A 100/0 weighting is not visibly different thanFIG. 5A, when rendered in the bitonal form of patent drawings; a 0/100weighting appears as a flat gray frame.)

While the techniques of blending/overlay with a watermark frame can beused with any web content, other arrangements can also be used. One suchalternative is next described, in the context of an online blog entry.

Blogging sites have gained enormous popularity since their introductionin the 1990s. The most popular at the time of this writing is WordPress,which serves here as an exemplary platform. Other popular blogging sitesemploy similar constructs. WordPress is familiar to those skilled in theart, so its operation is not much belabored.

One of the features that makes WordPress popular is its ease of use.Even those with nil experience in web authoring can put together aWordPress blog. In large part, this is due to its use of “templates”that define standardized structural features of the web presentation. Byselecting different templates, a user can quickly re-cast the user'scontent into different forms, with various layouts, colors, fonts, etc.Collections of such features are termed a WordPress “theme.”

To create a blog using WordPress, a user is prompted to pick a theme,and next enter text and pictures. WordPress then automatically authorsand publishes a web page, with the user-submitted text and photospresented in a layout defined by the theme.

FIG. 6A shows such a web page 60. This theme includes a user-selectedbanner artwork 62 at the top of the page, and user-entered text 64 in ablog entry 65. (As is familiar, such a blog site typically presentsseveral blog entries—commonly arranged by date or topic. Only one blogentry 65 is shown in FIG. 6A. If the blog entry is more than a paragraphor two, the page 60 commonly presents just the first part of the blogentry, with a “Read More” link that causes the web browser to load a newweb page where the blog entry is presented in unabridged form.)

The remainder of the displayed web page is established by theuser-selected theme, i.e., data stored in the corresponding template.This includes the size and font of the type for the user-entered text64, the rectangular shape of the blog entry 65, the positioning of theblog entry relative to the banner artwork, a first textured pattern 66that serves as a background for the blog entry (over which the blog text64 is presented), and a second textured pattern 68 that serves as abackground for the web page (over which the blog entries 65 arepresented). There is a considerable amount of software instructions(e.g., HTML and the like) in the template that defines all of thislayout and structure, but such detail is not typically exposed to theuser.

In accordance with another aspect of the present technology, thebackground pattern(s) in such a web page is digitally watermarked.(Indeed, such a pattern may comprise a pure watermark pattern—with itsrandom-seeming encoding of machine-readable data amplified to a levelwhere it serves as a visible and aesthetically interesting texturepattern. Alternately, the watermark may be applied to a practically flatbackground, which appears to the user not to carry data, except underclose scrutiny.)

In FIG. 6A, the pattern 68 that serves as the background to the web pageis watermarked with payload data that causes a smartphone, when itcaptures an image of such pattern, to link to the blog web page. (One ofthe component blocks of this second watermark pattern 68 is outlined asitem 69 in FIG. 6A.) The pattern 66 that serves as the background to theblog entry is watermarked with payload data that causes a smartphone,when it images such pattern, to link to the expanded blog entry (i.e.,the expanded entry that results when the user clicks the “Read More”link at the bottom left of the blog entry 65).

As with the earlier-discussed examples of FIGS. 1A-C, and 5A-B, thephysical size of the presented web page 60 of FIG. 6A will varydepending on the size of the display device, the size of the displaywindow, and any zoom parameter with which the web page is viewed.

The FIG. 6A web page is displayed at a zoom level of 90%. FIG. 6B showsthe same web page, with a zoom level of 125%. Note that the bannerartwork 62, the text 64, and the size of the blog entry 66, are allproportionately larger.

But also note that the pattern 68, and the component block 69, arepresented at the same scale in both FIGS. 6B and 6A. Although not asevident, the background pattern 66 is also presented at the same scale.These are the watermark patterns, whose scale is maintainedsubstantially constant, despite zoom level, window size, and physicalsize of the display device.

FIGS. 7A-7C show one way of achieving such behavior, for the GoogleChrome and Microsoft Explorer browsers.

FIG. 7A shows an excerpt of the CSS instructions for the web page ofFIGS. 6A and 6B. As noted, CSS instructions are commonly used to definethe look and formatting of documents authored in a markup language(e.g., XML, HTML).

The particular CSS instructions shown in FIG. 7A define a backgroundwatermark pattern, termed wm1. The background is characterized by a URL(which points to a data repository that stores data defining a watermarkpattern block), and a size of the component block (in pixels). This URLand size data are inserted between the paired /* and */ markings in theFigure. (While the URL may point to a remote machine, it need not; theblock pattern may be stored on the user's machine.)

Similarly, the CSS instructions include parameters for the div where thebackground watermark will appear.

A div is an HTML construct by which online content can be divided (thusthe name) into different parts, e.g., to which different formatting canbe applied. In FIG. 6A, the rectangular area of the depicted blog entry65 is one such div. The watermark information just-noted forms part ofthis div's definition. FIG. 7A indicates that other parameters of thediv, such as its width and height, are also detailed in the CSSinstructions (again, inserted between the /* and */markings).

FIG. 7B shows JavaScript instructions defining a function redraw( ).These instructions start by defining variables desiredTileWidth, wmDiv,and ratio.

The first of these, desiredTileWidth, is the size at which the watermarkblocks should be maintained, in pixels. (Again, the parameter isinserted between the /* and */.) The second, wmDiv, establishes thespecific div that contains the watermark blocks as background. Thisvariable is set equal to the value document.getElementByID(“wm1”)—a W3Cconvention for obtaining a reference to an element by its ID.

The third variable, ratio, is a factor by which the watermark blockshould be changed in size. As will be apparent, here the block size ischanged based on the zoom factor (or display size or window size) so asto maintain a substantially fixed block size in all cases. Inparticular, the JavaScript code of FIG. 7B defines the variable ratio tobe a value equal the fraction window.innerWidth divided bywindow.outerWidth.

Window.innerWidth is the pixel width of the content being displayedwithin a browser window. If a user zooms-in on the content, a smallerpixel width-portion of the content will be displayed (albeit at agreater size), so this value diminishes. Window.outerWidth, in contrast,is the pixel width of the browser window. This value does not vary withzoom factor.

This may be made clearer by reference to FIGS. 8A-8C. All of thesefigures show an identically-sized window, having a window.outerWidthvalue of 1440 pixels. Each shows four lines of text as the displayedcontent. (The arrows are not part of the displayed content; nor are thevertical and horizontal lines that serve as graticule markings.)

The top text line in each window gives two values of “width.” The firstis the pixel size of the content displayed within the window (i.e.,window.innerWidth); the second is the pixel size of the window (i.e.,window.outerWidth, or 1440).

In FIG. 8A, the content is displayed at 100% scale. Both of the widthparameters are 1440.

In FIG. 8B, the content is zoomed-out, with a zoom value of 90%. As thewindow is zoomed-out, the size of the content shrinks. Thus, the textshown in FIG. 8B is smaller than in FIG. 8A. (This is illustrated by theoverlaid black arrow, which points to the colon in the last line oftext; note that it shifted to the left between FIGS. 8A and 8B.) Becausethe content is physically reduced in size, more of it fits within thedepicted window. Thus, the first of the width parameters (in the firstline of text)—the pixel width of content displayed within the window—isnow 1600, instead of 1440.

FIG. 8C shows the opposite situation, with the content zoomed-in, hereat a value of 110%. Reciprocally, the text shown in FIG. 8C is largerthan in FIG. 8A. The black arrow (indicating the colon in the last nextline) has shifted to the right. Because the content is enlarged in size,less of it fits within the window. Thus, the first of the widthparameters (window,innerWidth) is now 1310, instead of 1440.

Each of FIGS. 8A-8C also shows, in the third line of text, the value ofthe ratio variable referenced in FIG. 7B. For FIG. 8A, it is 1; for FIG.8B, it is 1.1111; and for FIG. 8C it is 0.9097.

This ratio is the factor by which the size of the watermark block needsto be changed, to counteract changes in size of the displayed contentdue to zooming. Thus, for example, in FIG. 8B, the displayed text hasshrunk in size. To prevent the watermark blocks from shrinking by asimilar amount, they must have an original scale of 1.1111 times theirnominal size. (The nominal size may be one inch on a side, e.g., 72pixels on a 72 dpi display, so the adjusted block should have a scale1.1111 times this value, or 80 pixels.) When the shrinking factorexhibited by the text is likewise applied to such a watermark block, itis presented at its nominal scale, i.e. one inch in width.

Desirably, the re-sized block of pixels that make up a watermark blockis an integral number of pixels in width. Due to this quantization to aninteger value, there will be some variability in the physical size ofthe watermark block, at different zoom values. In the examplejust-given, the block will be 80 pixels on a side for zoom valuesbetween 1.1042 and 1.119, leading to blocks that vary from 0.9938″ to1.0062″ in width. This is one of the reasons that a “substantially”consistent watermark scale is referenced across different zoom values,instead of an absolutely constant watermark block size.

Returning to FIG. 7B, it will be seen that the variable ratio is defineddifferently if the JavaScript is running on Microsoft's InternetExplorer (IE) browser. In this case, ratio is defined as the ratio ofscreen.logicalXDPI to screen.deviceXDPI. This approach is required dueto a variant meaning given to window.outerWidth by IE; Microsoft's XDPIproperties provide a work-around.

Continuing with FIG. 7B, the JavaScript instructions next define thevariable size as the product of the earlier-referenced desiredTileWidth,and the just-discussed ratio. For example, the variable size has a valueof 80 in the just-discussed example.

Continuing, the variable styleString is defined as the concatenation ofthe variable size (i.e., 80) with the text “px” concatenated again withsize and the text “px.” So in the example just given, this variablestyleString has a value “80px 80px”.

Finally, the last line in FIG. 7B assigns this just-discussed string towmDiv.style.backgroundSize.

FIG. 7C shows HTML that implements the just-discussed redraw( )function.

These instructions begin by calling the redraw( ) function whenever anonresize event is detected. (This occurs whenever the browser window isresized.) So whenever the window is resized, the redraw( ) functioncomputes the pixel dimensions of the watermark block, ensuring that itpresents at a substantially consistent scale on the display.

Likewise, the FIG. 7C code calls the redraw( ) function on detection ofan onload event (i.e., when a web page finishes loading). Thus, when anew web page is displayed, the JavaScript code will be called toestablish the correct size of the watermark blocks.

The FIG. 7C code also establishes that these actions are to apply to thediv identified by the ID “wm1.”

Again, the code of FIGS. 7A-7C is employed in connection with thetemplate used to present the blog page of FIG. 6A. The wm1 divcorresponds to the blog entry 65, so this code serves to define and sizethe watermark blocks of background pattern 66. (Similar code effectssimilar functionality for the div that spans the body of the blog,including watermark background pattern 68.)

The just-discussed code is exemplary. The same or similar functionalitycan be implemented in a number of different manners (including, e.g.,JQuery, Oracle Java, and Adobe Flash software), so this example codeshould in no way be regarded as limiting.

By an arrangement like that just-described, a user can select a templatethat includes associated watermarking instructions, to thereby yield aweb page with watermark-based functionality—without any effort orexpertise concerning digital watermarking.

(For use with the Firefox browser, the CSS instructions of FIG. 9 areemployed. These instructions employ Mozilla's “dppx.” This unitrepresents the number of dots per px unit. Due to the 1:96 fixed ratioof CSS in to CSS px, 1 dppx is equivalent to 96 dpi, which correspondsto the default resolution of images displayed in CSS as defined byimage-resolution.)

FIG. 10 gives a basic overview of certain linking aspects of the presenttechnology. A first user with a smartphone 91 captures an image ofcontent 92 presented on a screen of a second user. In this example, thecontent is imagery of the Goat Rocks Wilderness from the web site of thePacific Crest Trail Association (PCTA), displayed in a Google Chromebrowser window. The code served from the web site, or code in thebrowser, scales watermark blocks based on the display size (e.g., asdiscussed in connection with FIGS. 7A-7C). CSS instructions cause thewatermark blocks to be overlaid on the PCTA imagery as a translucent“div,” as depicted in connection with FIGS. 5A and 5B.

App software in the smartphone 91 processes the captured imagery andextracts the payload encoded by the digital watermark pattern. In thepresent case, the payload is the hex string 2A34A.

The smartphone wirelessly transmits the payload data, and it is receivedby a remote server computer 93. The server includes database software,here depicted as a table 94, which associates different watermarkpayloads with different URLs. The 2A34A watermark payload decoded by thesmartphone 91 resolves, through the table, to a URL at GoogleMaps—detailing the area in the Goat Rocks Wilderness depicted by thecaptured image. This URL is sent back to the first user's smartphone 91,where that Google Maps content is presented by the browser on thesmartphone.

Naturally, the watermark payload can link to any URL (or other code ormetadata). In many implementations, the URL will point to the contentoriginally captured by the smartphone camera from the second user'sscreen, e.g., the photo 92 depicting the Goat Rocks Wilderness.

In some applications, several URLs can be associated with a singlepayload in the table 94. A menu can then be presented on the smartphonescreen detailing the different link choices, any of which the user maychoose (e.g., a link to the original image 92, a link to the Google Mapspage for the Goat Rocks Wilderness, a link to the Wikipedia page for theGoat Rocks Wilderness, etc.). Alternatively, stored rule data (either atthe server 93, or in the smartphone 91) can be applied to automaticallyselect between the available links.

In many implementations, the link returned to the smartphone will bedetermined in part based on context, such as information about the user.So a retired pensioner in Europe, capturing an image of the Goat RocksWilderness from another user's screen, will be routed to a different URL(e.g., the Wikipedia page) than a college student in Yakima, Wash. (whomay receive, e.g., driving directions for the 20 mile trip).

Similarly, if a college student in Atlanta navigates to a web site forthe Florida Department of Tourism, the web page may be dynamicallyCSS-marked with a watermark payload corresponding to an advertisementfor Spring Break beachfront hotel packages. In contrast, if a 70 yearfrom Atlanta navigates to the same web site, the page may be dynamicallymarked with a watermark payload leading to promotions for golfingpackages.

In some embodiments, the watermark payload extracted from the content 92may lead to presentation of several items of content on smartphone 91.For example, the original image 92 may be depicted together with abottom border that contains one or more watermarks that link toauxiliary content on the receiving smartphone, such as the noted map andWikipedia pages, which the user can then choose between. Alternatively,the border may contain a banner ad for the backpack depicted in theimage (e.g., which watermark-links to a corresponding page on the website of The North Face outfitter).

As is familiar, ad placement on web pages is commonly performed byso-called “Supply Side Platforms”—software tools that discerninformation about a user who visits a web page (typically through use ofcookie data), and determine which ad should fill an ad slot available inthe web page delivered for that user's visit. In many instances, thesoftware conducts a quick online auction to identify the vendor willingto pay the most. Such technology is familiar from the Google AdWordservice. The AdWord service selects an advertisement for presentationwith Google search results, based on the user context (e.g., demographicinformation and browsing history), and on the user's search query.Related ad serving technology is further detailed in publishedapplication 20140164111, and is likewise suited for use with the presenttechnology.

In the arrangements just discussed, the data structure 94 may associateseveral different items of auxiliary content with a single watermarkpayload (and, e.g., the server or the smartphone can select between thembased on rules and/or context information).

In another arrangement, the watermark blocks presented with content 92can adapt to context, and convey different payloads. For example, theCSS instructions in FIG. 7A can select among different watermark blocksthat should be presented with the content, based on data available tothe system on which the content is being presented. To give a simpleexample, code on the device originally presenting the content 92 canquery its operating system to provide the IP address of the device onwhich the content is presented. Using known techniques, the IP addresstypically allows the location of the device (e.g., metro Seattle), andthe internet service provider (e.g., Comcast), to be identified. Thecontent 92 can be presented with a watermark that is a function not onlyof the content, but also of the location and the ISP. (Indeed, in someinstances, resolution of the watermark using database 94 may identifythe location and ISP.) Similarly, any context information discernable bythe device can be used in selecting the watermark payload that should bepresented. (Again, rule data can naturally be employed.)

Additional information about the retrieving and using links (and othermetadata) based on watermark payloads extracted from content, isdetailed in U.S. Pat. No. 8,364,720.

More on Blogs

WordPress charges a monthly fee to users who want enhanced bloggingservices. In a previous online marketing paradigm, this cost might havebeen reduced, or waived, if a user agreed to allow banner adpresentation with their blog. In accordance with another aspect of thepresent technology, such discount is provided to the user withoutpresentation of banner advertising on the user's blog site. Instead, theblog is marked with a steganographic watermark that conveys a link toadvertising material.

In such an arrangement, the hosting service (e.g., WordPress) alertscandidate advertisers to different blog pages, identifying the trafficeach is drawing (including cookie data, where available), and keywordsabout the content displayed on the blog. If one blog page has ablogger's tribute to Jimmy Choo boots, and is found to be drawing alarge viewership of 18-30 year old women, this information can berelayed (e.g., using known ad serving arrangements) to candidateadvertisers. Nordstrom may offer a nickel for each viewer impression;Zappos Shoes may offer a dime. WordPress sells the rights to Zappos, andserves the blog page with code that causes a watermark to be presented.WordPress enters data in the database 94 causing this watermark to linkcode that both displays the blog content, and also presents a Zappospromotion for Jimmy Choo boots. Whenever any user captures an image ofthe blog with their smartphone, the smartphone app loads content fromthe blog and presents it on the smartphone, together with the Zappospromotion. Each time such a watermark is resolved by the database,Zappos pays WordPress a dime. (Or each time the Zappos promotion isserved and displayed by the viewer's smartphone, Zappos pays a dime.)

In another arrangement, watermark-based linking is another enhancedfeature for which the blogging service can charge a user. For example, ablogger may post images and travelogue text relating to a recent trip toItaly, but may want to provide a richer content experience than theblogger has time to author. In such case, the blogger may pay a fee, inexchange for which an automated ghostwriter algorithm spices up the blogby defining watermark-based hyperlinks for the user's content. Forexample, if the blogger posted a picture of a calamari dish enjoyed at aVenice restaurant, the ghostwriter may link such picture to the web sitefor the restaurant, or its online menu, by data entered in a database 94in association with the watermark payload for that picture.

Display-Device Pairing

Display devices are becoming ubiquitous, and a great number are found inpublic spaces. Smartphones and wearables are highly capable computers,but one of their drawbacks is their small display size.

In accordance with another aspect of the technology, a user with acamera-equipped device (and optionally an external keyboard, such as aBluetooth keyboard) captures an image of a public display screen. Thesoftware code that is controlling the screen applies a watermark patternto whatever information is on the display, using the techniques detailedherein. This watermark pattern resolves, such as through a database 94,into an IP address to which the device can transmit data for display onthe screen.

Library carrels at universities, for example, can be equipped with suchdisplay devices. A student may sit down and, using a smartphone camera,wirelessly pair the smartphone to the display. With an accessorykeyboard the student can have the functionality of a desktop computer,but without the weight, and without fussing with cables. Moreover, thestudent's information (cookies, etc.) is not shared with the display orits associated processor; only the data needed for display istransmitted from the smartphone device.

Such pairing can alternatively be achieved through other data signalingtechnologies, including audio (such as an ultrasonic data signal or alow amplitude noise signal encoding the data), barcode, RFID/NFC, etc.Each can convey a payload that resolves—through a database—to an IPaddress to which display data for that screen device can be sent.

By such arrangements, a small device can easily cast its output to anydisplay screen, increasing the utility of the former, and the use of thelatter.

Other Arrangements

A further use of the present technology is for play-out monitoring,e.g., to confirm that correct content is playing on correct publicdisplays at correct times.

Consider an airport, which may have plural electronic advertisingsignboards positioned along a concourse walkway. A ceiling-mountedsurveillance camera with a wide angle lens may capture imagery from oneor more of the signs. The location of the camera, and each of the signs,is known.

Rather than watermarking, at the time of content creation, each item ofHTML advertising content that is to be displayed on such a screen, andhoping that the encoded watermark scale permits reading from asurveillance camera viewing that screen, the present technology can beemployed. That is, the advertising can be created without regard towatermarking, but at the time of rendering, a watermark is applied. Andthe tile size for the displayed watermark (as well as its payload) canbe established based on the screen's location.

Imagery captured by a surveillance camera can be analyzed to determinewhat content was presented on each sign at different times. Imagefingerprinting techniques can be employed to identify the content (withthe watermark simply indicating the screen from which the imagery wascaptured). Alternatively, the watermark selected for inclusion atrendering may serve to uniquely identify the content. (For example, theHTML instructions, or a particular tag therein (e.g., “<TITLE>”), can behashed to generate an identifier that uniquely identifies that content.This identifier can index a data structure, which identifies acorresponding watermark tile that should be selected for inclusion inthe rendered output. When that watermark is later detected, the samedata structure can be consulted to determine the content hash, andthereby identify the content.)

The camera-captured imagery can be processed to effect some grossgeometric correction (e.g., warping the imagery so that the rectangularscreen is presented with approximately its correct aspect ratio, ratherthan with a perspective-skewed shape), before watermark decoding (andoptionally finger-based content identification) is performed.

In some embodiments, each screen and camera is equipped withlocation-determining technology (e.g., GPS), so that each screen canautomatically select an appropriate watermark scale based on a computeddistance to the nearest camera (and on its own display dimensions).

Relatedly, the ability of the present technology to present a watermarkreadable from a particular distance allows beamforming-likeapplications, with different watermarks targeted to be decodable byviewers at different positions relative to a screen.

A simple example is a Times Square billboard that is programmed tosplash different payloads to viewers at different distances—eachyielding a different payoff. For viewers near such a sign, on one sideof a street, the operator may provide watermark-triggered coupons tostores on that side of the street. For viewers remote from thesign—across the street, the operator may provide coupons to stores onthat opposite side of the street.

Moreover, the code of FIGS. 7A-7C can be modified so that the width andheight of the watermark tiles are separately modified, unequally. Thisallows the watermark to be pre-distorted, to compensate for distortionfrom off-axis viewpoints (i.e., viewpoints off the axis extendingperpendicularly from the center of the screen). That is, a tile can bepre-distorted into a non-square shape such that, when viewed from anintended viewer's viewpoint, each tile appears square and of identicalsize.

A proprietary browser (or WebGL, such as used in Google Maps) can thusrender a display in which the watermark tiles target (i.e., aredecodable) by differently-located viewers at different times (e.g., eachrefresh of the screen targets a different viewing location, even if theunderlying imagery is static).

(Related technology is detailed in applicant's U.S. Pat. No. 8,412,577,which describes how electronic billboards and other public displays canbe operated to cyclically present watermarks at different scales, tofacilitate detection by camera-equipped smartphones at differentviewpoints.)

In some such embodiments, data can be passed to the FIG. 7B (or other)software, detailing parameters such as the viewing distance, opticalparameters of a viewing camera (e.g., zoom, angular field of view,sensor dimensions in pixels, etc.), off-axis viewpoint angles (e.g.,azimuth and elevation). The watermark tiles added during rendering canthen be tailored in accordance with such information.

Another use of the present technology is at retailers of televisions andother screen-based devices. A Best Buy store may have dozens oftelevisions on display. The manager may periodically move televisions asnew ones become available and older ones are discontinued.

Retailers are increasingly using in-store cameras to monitor customertraffic, and dwell times of customers at different locations within astore—generating heat map data to identify which products are gettingthe most attention. This heat map information is of limited informationif it is uncertain what products are presently at positions wherecustomers are dwelling.

Certain embodiments of the present technology overcome this uncertainty.Each television can present a different watermark overlay signal on thedemo TV imagery, using the technology detailed herein. The scale of thewatermark is tailored, e.g., based on the display size. The trafficanalysis cameras (or other cameras) capture imagery including thepresented watermarks. Watermarks are decoded from this imagery, andserve to identify the different televisions at different positions inthe store (e.g., the 60 inch Samsung UN60H6350 TV is here, the 32 inchVizio E320AR TV is there, etc.). Customer heat map data can then berelated accurately to the products that draw customer attention.

Still more applications arise as wearable computers proliferate.Consider a customer in Best Buy wearing the Google Glass computer, orother such camera-equipped headworn device. Each screen the customerencounters may be overlaid with a different watermark, which is decodedby the wearable computer and stored in association with a time-stamp.(Watermarks decoded from printed signage, product packaging, etc., canbe handled similarly.) These decoded watermarks serve as waypointsdetailing the customer's path through the store, and dwell time atdifferent points. (In a sense, the log of sensed watermark informationserves like a cache of web cookies—detailing a user's visits toparticular locations.) The customer may be incented (by coupons orotherwise) to share this log of waypoints with Best Buy. Or a competingretailer may make the customer a better offer to share this sameinformation. (The same arrangement can be practiced with informationabout sensed audio watermarks or ultrasonic audio, where differentwatermark payloads/ultrasonic audio are used at different locations in astore.)

Wearable computers (such as Glass) are commonly equipped with positionand motion sensors, such as GPS, and 3D-magnetometers, -gyroscopes, and-accelerometer sensors. If the position of an electronic screen is known(e.g., latitude/longitude and compass direction it faces), and the sameinformation is known for the camera of the wearable computer, then alsoknown is the relative geometry, which defines the perspective with whichthe camera views the screen, and establishes the resulting distortion ofthe view. The wearable can transmit its position information (e.g., byBluetooth, shortrange ultrasonic audio, etc.), and the store can usethis information in changing the geometric presentation (e.g., the scaleand affine pre-distortion) of the watermark on screens within thecamera's view. Additionally, or alternatively, the store can similarlytransmit a screen's position information, and the wearable can use thisinformation in de-warping imagery captured by the camera so thatwatermark information can be more easily decoded therefrom.

Still further, a store system with a fixed camera (e.g., on the ceiling)can use image classification techniques to determine whether a shopperdepicted in the camera's imagery is or is not wearing a headworncomputer. If the shopper is wearing such an apparatus, imagery from thefixed camera can be analyzed to estimate the position and orientation ofthe headworn computer (substituting for information transmitted from thewearable, in the preceding paragraph). Watermarks in displays within 20feet of the shopper can then be properly pre-distorted for decoding bythe shopper. (The perceptible image can also be pre-distorted in thisfashion, yielding something akin to a “forced perspective”display—providing an illusion that the viewer is looking at the screen'spresented artwork head-on, instead of from an oblique perspective.)

While the focus of this disclosure has been on screen displays,principles thereof are also applicable to projected displays.

Projected displays are familiar from the Kinect home gaming system,where they are used to perform 3D sensing and gesture detection.Applicant believes they also have utility in retail stores, e.g., wherethey can project information for reading onto store products andshelves.

The Kinect system is a form of structured light scanner, using arandom-looking array of infrared projected dots. In accordance with thepresent technology, the projected pattern (which may be infrared, ornot) encodes payload information. Different payloads can be encoded indifferent spatial regions of the projected field. Watermark, barcode, orother forms of machine-readable data encoding can be employed.

These payloads can correspond to different products, and the locationsto which these payloads are projected may indicate where differentproducts should be shelved. Cheerios cereal should be found in theselocations; Wheaties cereal should be found in those locations; etc.Store “planograms” commonly are used to specify what products (typicallyidentified by their respective “SKU”s) should be shelved where.Planogram data can serve as the input from which the project pattern isderived, with each differently-encoded region corresponding to adifferent SKU.

The projected patterns can be sensed by a smartphone or othercamera-equipped device conveyed by a store clerk, and used in stockingthe store shelves. While the projector may form part of the camera unitcarried by store clerk (with the projected pattern dynamically updatedbased on sensed location of the device), the projector may more often befixedly mounted to the store infrastructure, such as mounted on theceiling or to a shelf. In this case, the pattern may be pre-distorted(i.e., before projection) so that it has an intended, undistortedappearance when viewed from the surface on which it is expected to beprojected, using a camera with an expected viewpoint. That viewpoint maybe looking horizontally at a store shelf, from the same elevation as theshelf. Or it may be looking at the shelf (up or down, depending on shelfheight) from an elevation of 40 inches. Etc. (FIG. 11 shows what happenswhen a uniform grid of dots is projected onto a surface (in this casecurved), and viewed from a different angle—without such pre-distortion.Reversing the role of camera and projector, FIG. 11 may also be viewedas a pre-distorted pattern that—if viewed from the other angle, wouldappear as a uniform grid of dots.)

In a particular embodiment, different pre-distortions are cyclicallyapplied (e.g., by suitable WebGL browser instructions that haveknowledge about the shape and location of the shelving), to permitreading of the projected SKU information from different viewpoints. Asplanogram information is revised, the projected pattern—derived from theplanogram—is similarly revised.

In checking stocked shelves for compliance with a planogram, someimplementations do not require the store clerk to judge if a product isout of place. Instead, camera data from which the SKU placementinformation is discerned can also be applied to an image recognitionengine (e.g., based on image fingerprinting and stored reference data)to identify what product is actually found in a given region of acaptured image frame. Or the product may be identified by decoding asteganographic digital watermark from its packaging. If the identifiedproduct does not match the corresponding projected SKU information, themis-placed product can be flagged. (E.g., a flashing red rectangle maybe overlaid around the depiction of the mis-placed product on the devicedisplay, in augmented reality fashion, and/or an audible alert may besounded to draw the clerk's attention to a shelving error.)

In some embodiments, the task of checking compliance with storeplanogram information is performed by a camera-equipped robot thattraverses store aisles at night. Its computer compiles a hit list ofitems needing correction, which can later be attended to by a storeclerk.

(Additional information about projectors in retail stores is detailed inapplicant's published application 20140052555.)

Concluding Remarks

Having described and illustrated the principles of our inventive workwith reference to illustrative examples, it will be recognized that thetechnology is not so limited.

For example, while the detailed embodiments focused on presentation ofHTML pages, applicant's inventive work can be applied to any electronicinformation (including games, videos, text, images, etc.), displayed inany manner.

Moreover, while the detailed embodiments focused on reading watermarkinformation from displayed content using a camera-equipped device, thesame principles can likewise be applied to decoding watermarked datadirectly from the digital content—without any need for image capture.

Certain of the examples given above employed a flat grey watermarkpattern. This is suitable for many embodiments. However, in others, itis desirable to adapt the strength (e.g., RMS amplitude) of thewatermark signal based on features of the underlying image content, toreduce watermark visibility.

Consider an image that includes grass in the foreground, and sky above.Since the grass imagery is highly random, the randomness introduced bythe watermark signal will not be noticed; a very strong watermark can beapplied in that area. In contrast, a strong watermark should not beapplied to the sky, lest the randomness of the watermark be conspicuousin the normally-flat texture of the sky.

In some embodiments of the technology, the watermark pattern accessed bythe URL in FIG. 7A can be tailored based on the image with which it isto be presented, to adapt its strength based on the data hiding capacityof the image. The FIG. 7A-7C code can be arranged to access a storedspatial mask that defines the strength at which the watermark should beapplied at different local regions of the image. Masks of differentscales can be obtained, based on the size of the presented content.

Instead of (or in addition to) locally adapting the strength of thewatermark to the host imagery, the chrominance of the watermark can beadapted so as to better hide within such content. Again, plural suchwatermark patterns can be pre-computed, based on different scalefactors, and a scale-appropriate one can be selected based on the scaleof the associated content.

More information about adaptation of an image watermark based on theunderlying host image content is provided in the watermarking referencescited herein.

In some implementations, CSS code can fetch plural different watermarkblocks—each encoding the same payload, but at different scales. Thesecan be cached in the local device, and switched-between, e.g. as thecontent is zoomed, or the display window is re-sized.

Although the technology has frequently been described as maintaining asubstantially consistent watermark block scale, this is not essential.Since watermark readers are commonly robust over a finite variety ofscale states, a relatively few watermark blocks of different scales canenable linking over a large range of display scales. For example, onewatermark block scale may be used for zoom values between 20% and 35%; asecond for values between 35% and 60%; a third between 60% and 100%; afourth between 100% and 170%; and a fifth between 170% and 290%.

In some embodiments, the instructions (e.g., of FIGS. 7A-7C) can specifywhat payload should be encoded in the watermark block. Such a parametercan be passed, e.g., by the CSS instructions to a remote server. Theserver can then create a watermark block which encodes the specifiedpayload on the fly, and return it (or its URL, etc.) to the calling CSScode (at the requested scale).

As suggested, the watermark payload (or metadata with which it isassociated in a data structure) can be of numerous types. Suchinformation can be arbitrary, or identify the user, or user demographicinformation, or the user's hardware, or the content being viewed, orauxiliary content related to the viewed content, or the date/time, orlocation (latitude, longitude, elevation), and/or compass orientation ofthe screen, etc., etc.

Artisans will recognize that different displays have different densitiesof pixels-per-inch. Early Apple products used 72 ppi (to correspond withthe “point” system of typesetters). Recent Retina displays by Appleemploy PPIs of between 220 PPI (for the MacBook Pro) up to 326 (for thesmallest devices, such as iPhones). A watermark block that is one inchwide, when rendered on an early Apple display, will render at less thana fourth of this size on an iPhone, due to the smaller physical size ofpixels in the latter screen. Many devices and software cope with thisvariability by assuming that the display has 96 pixels per inch, forpurposes of compatibility. Interpolation can be used to fill pixelsbetween these 96 PPI locations.

While functions window.innerWidth, window.outerWidth,window.innerHeight, and window.outerHeight, were particularly mentioned,the present technology can use other functions, includingdocument.body.clientWidth and window.screen.availWidth.

Moreover, various hacks can be employed to determine the physical pixeldensity of the display device (e.g., requesting the user to zoom areference image of known pixel dimensions until it fills the width of abrowser window, and then using the window,innerWidth andwindow.outerWidth functions to deduce the physical pixel width).Alternately, various smartphone operating systems and softwaredevelopment kits make available methods that can be used to query thecurrent device for the scale at which it displays content, for its widthand height in pixels, etc. Still other such functions can be used inobjective C, Java, etc. By such arrangements, the scale of watermarkblocks can be maintained substantially consistent, across displaydevices having diverse pixel densities, and across differenthardware/software environments.

Principles of the present technology can be applied in situations thatmake no use of watermarking. For example, a pattern that serves as abackground for a web page can be identified by fingerprinting, e.g.,using SIFT points. A data structure can again correlate differentbackground patterns to different content, such as URLs and advertising.(The scale of such a pattern can be maintained at a fixed size,irrespective of the scale of the primary content, although this may beless important with fingerprinting.)

The displayed content need not be presented for viewing on an electronicscreen. For example, it can be projected onto a passive surface.Similarly, while repeated reference was made to web pages, it will berecognized that such pages needn't be delivered over the internet.

Moreover, principles of this technology can be applied in connectionwith content rendered in a non-electronic form, such as printed onpaper. Constructs analogous to the W3C structures exist for printrendering, e.g., the PostScript page description language, and can beemployed in similar fashion. (Applicant's U.S. Pat. Nos. 6,899,475 and7,142,691 also concern watermarking using page description languages.)

While FIG. 10 shows a simple lookup table as the database 94, moreelaborate data structures will likely be used in actual practice. Thismay include federated databases—employing information stored atdifferent locations by different parties.

The artisan will recognize that translucency is supported by the RGBAform of image data, where the “A” channel (alpha) indicates the degreeof opacity with which such imagery is to be applied to the displaydevice.

Although reference was made to app software on a smartphone asperforming certain of the detailed functionality, it will be recognizedthat these functions can naturally be performed otherwise—including byoperating system software on a smartphone, by smartphone hardware, by aserver (e.g., at a social networking service), by another smartphone orcomputer device, by dedicated hardware, distributed between suchdevices, etc.

While reference has been made to smartphones, it will be recognized thatthis technology finds utility with all manner of devices—both portableand fixed. Tablets, laptop computers, digital cameras, wrist- andhead-mounted systems and other wearable devices, servers, etc., can allmake use of the principles detailed herein. (The term “smartphone”should be construed herein to encompass all such devices, even thosethat are not telephones.) Particularly contemplated smartphones includethe Apple iPhone 6; smartphones following Google's Android specification(e.g., the Galaxy S4 phone, manufactured by Samsung, and the Google MotoX phone, made by Motorola), and Windows 8 mobile phones (e.g., the NokiaLumia 1020, which features a 41 megapixel camera).

Details of the Apple iPhone, including its touch interface, are providedin Apple's published patent application 20080174570.

The design of smartphones and other computers employed in embodiments ofthe present technology is familiar to the artisan. In general terms,each includes one or more processors, one or more memories (e.g. RAM),storage (e.g., a disk or flash memory), a user interface (which mayinclude, e.g., a keypad, a TFT LCD or OLED display screen, touch orother gesture sensors, a camera or other optical sensor, a compasssensor, a 3D magnetometer, a 3-axis accelerometer, a 3-axis gyroscope,one or more microphones, etc., together with software instructions forproviding a graphical user interface), interconnections between theseelements (e.g., buses), and an interface for communicating with otherdevices (which may be wireless, such as GSM, 3G, 4G, CDMA, WiFi, WiMax,Zigbee or Bluetooth, and/or wired, such as through an Ethernet localarea network, etc.).

The processes and system components detailed in this specification canbe implemented as instructions for computing devices, including generalpurpose processor instructions for a variety of programmable processors,such as microprocessors (e.g., the Intel Atom, the ARM A5, the QualcommSnapdragon, and the nVidia Tegra 4; the latter includes a CPU, a GPU,and nVidia's Chimera computational photography architecture), graphicsprocessing units (GPUs, such as the nVidia Tegra APX 2600, and theAdreno 330—part of the Qualcomm Snapdragon processor), and digitalsignal processors (e.g., the Texas Instruments TMS320 and OMAP seriesdevices), etc. These instructions can be implemented as software,firmware, etc. These instructions can also be implemented in variousforms of processor circuitry, including programmable logic devices,field programmable gate arrays (e.g., the Xilinx Virtex series devices),field programmable object arrays, and application specificcircuits—including digital, analog and mixed analog/digital circuitry.Execution of the instructions can be distributed among processors and/ormade parallel across processors within a device or across a network ofdevices. Processing of data can also be distributed among differentprocessor and memory devices. Cloud computing resources can be used aswell. References to “processors,” “modules” or “components” should beunderstood to refer to functionality, rather than requiring a particularform of implementation.

Software instructions for implementing the detailed functionality can beauthored by artisans without undue experimentation from the descriptionsprovided herein, e.g., written in C, C++, Visual Basic, Java, Python,Tcl, Perl, Scheme, Ruby, JavaScript, HTML5, etc., in conjunction withassociated data. Smartphones and other devices according to certainimplementations of the present technology can include software modulesfor performing the different functions and acts.

Known browser software, communications software, imaging software, andmedia processing software can be adapted for use in implementing thepresent technology.

Software and hardware configuration data/instructions are commonlystored as instructions in one or more data structures conveyed bytangible media, such as magnetic or optical discs, memory cards, ROM,etc., which may be accessed across a network. Some embodiments may beimplemented as embedded systems—special purpose computer systems inwhich operating system software and application software areindistinguishable to the user (e.g., as is commonly the case in basiccell phones). The functionality detailed in this specification can beimplemented in operating system software, application software and/or asembedded system software.

Different of the functionality can be implemented on different devices.For example, in a system in which a smartphone communicates with acomputer at a remote location, different tasks can be performedexclusively by one device or the other, or execution can be distributedbetween the devices. Extraction of watermark information from imagery isone example of a process that can be distributed in such fashion. Thus,it should be understood that description of an operation as beingperformed by a particular device (e.g., a smartphone) is not limitingbut exemplary; performance of the operation by another device (e.g., aremote server), or shared between devices, is also expresslycontemplated.

In like fashion, description of data being stored on a particular deviceis also exemplary; data can be stored anywhere: local device, remotedevice, in the cloud (on a remote server), distributed, etc.

As noted, the present technology can be used in connection with wearablecomputing systems, including headworn devices. Such devices typicallyinclude one or more sensors (e.g., microphone(s), camera(s),accelerometers(s), etc.), and display technology by which computerinformation can be viewed by the user—either overlaid on the scene infront of the user (sometimes termed augmented reality), or blocking thatscene (sometimes termed virtual reality), or simply in the user'speripheral vision. A headworn device may further include sensors fordetecting electrical or magnetic activity from or near the face andscalp, such as EEG and EMG, and myoelectric signals—sometimes termedBrain Computer Interfaces, or BCIs. (A simple example of a BCI is theMindwave Mobile product by NeuroSky, Inc.) Exemplary wearable technologyis detailed in patent documents U.S. Pat. No. 7,397,607, 20100045869,20090322671, 20090244097 and 20050195128. Commercial offerings, inaddition to the Google Glass product, include the Vuzix Smart GlassesM100, Wrap 1200DX, and Star 1200XL systems. An upcoming alternative isaugmented reality contact lenses. Such technology is detailed, e.g., inpatent document 20090189830 and in Parviz, Augmented Reality in aContact Lens, IEEE Spectrum, September, 2009. Some or all such devicesmay communicate, e.g., wirelessly, with other computing devices (carriedby the user or otherwise), or they can include self-contained processingcapability. Likewise, they may incorporate other features known fromexisting smart phones and patent documents, including electroniccompass, accelerometers, gyroscopes, camera(s), projector(s), GPS, etc.

The artisan is presumed to be familiar with the web authoring constructsemployed in the illustrative examples of the present technology,including CSS, JavaScript, HTML5, and other W3C standards.

References were made to context. The artisan will understand thatcontext refers to any information useful in characterizing the situationof an entity (an entity being a person, place or object that isconsidered relevant to an interaction between a user and an application,including the user and application themselves).

Context information can be of many sorts, including computing context(network connectivity, resource availability, processor type, CPUcontention, etc.), user context (user profile, location, actions,preferences, nearby friends, social network(s) and situation, etc.),physical context (e.g., lighting, noise level, traffic, sensed sounds,recognized speech, etc.), temporal context (time of day, day, month,season, etc.), history of the above, etc.

Another taxonomy of context progresses from simple and concrete, tocomplex and abstract, starting with location, then physical context (asdetermined by sensors, e.g., device orientation and motion, temperature,infrared, video, 3D ambient audio, ultrasonic, humidity, gases and otherchemical), then user or device actions (e.g., writing, talking, reading,searching, navigating, pointing), then proximities (e.g., to people,vehicles, buildings, perimeters, jurisdictions, other devices), thensomatic (e.g., live datastreams of biometric information), then datafeeds (e.g., subscriptions and RSS feeds, social network follows, alertsand updates), then emergent live data (e.g., from external sources, suchas calls, text, email, weather forecasts), and finally n-dimensionalcontext history—encompassing some or all of the foregoing.

Context arrangements suitable for use with the present technology arefurther detailed in the documents incorporated by reference.

The techniques of digital watermarking are presumed to be familiar tothe artisan. Examples are detailed in documents identified above, aswell as in patent documents U.S. Pat. Nos. 6,614,914, 6,122,403,20100150434, 20110274310 and 20140052555. Such watermarks are commonlyimperceptible, meaning they are not noticeable to a viewer examiningwatermarked content under typical viewing conditions (e.g., officelighting, at a distance of 20 inches).

In addition to the documents identified above, applicant's other workthat is relevant to the present technology includes that detailed inpending patent application Ser. No. 14/244,287, filed Apr. 3, 2014,62/044,736, filed Sep. 2, 2014, and published patents and applicationsU.S. Pat. No. 8,516,533, 20120218444, 20120046071, 20130311329,20130314541, 20120284012 and 20140106710.

This specification has discussed several different embodiments. Itshould be understood that the methods, elements and concepts detailed inconnection with one embodiment can be combined with the methods,elements and concepts detailed in connection with other embodiments.While some such arrangements have been particularly described, many havenot, due to the number of permutations and combinations. Applicantsimilarly recognizes and intends that the methods, elements and conceptsof this specification can be combined, substituted and interchanged—notjust among and between themselves, but also with those known from thecited prior art. Moreover, it will be recognized that the detailedtechnology can be included with other technologies—current andupcoming—to advantageous effect. Implementation of such combinations isstraightforward to the artisan from the teachings provided in thisdisclosure.

While this disclosure has detailed particular ordering of acts andparticular combinations of elements, it will be recognized that othercontemplated methods may re-order acts (possibly omitting some andadding others), and other contemplated combinations may omit someelements and add others, etc.

Although disclosed as complete systems, sub-combinations of the detailedarrangements are also separately contemplated (e.g., omitting various ofthe features of a complete system).

While certain aspects of the technology have been described by referenceto illustrative methods, it will be recognized that apparatusesconfigured to perform the acts of such methods are also contemplated aspart of applicant's inventive work. Likewise, other aspects have beendescribed by reference to illustrative apparatus, and the methodologyperformed by such apparatus is likewise within the scope of the presenttechnology. Still further, tangible computer readable media containinginstructions for configuring a processor or other programmable system toperform such methods is also expressly contemplated.

To provide a comprehensive disclosure, while complying with thestatutory requirement of conciseness, applicantincorporates-by-reference each of the documents referenced herein. (Suchmaterials are incorporated in their entireties, even if cited above inconnection with specific of their teachings.) These references disclosetechnologies and teachings that can be incorporated into thearrangements detailed herein, and into which the technologies andteachings detailed herein can be incorporated. The reader is presumed tobe familiar with such prior work.

The claims submitted with this application address just a small fractionof the patentable inventions disclosed herein. Applicant expects manymore, and broader, claims will be issued from this patent family.

In view of the wide variety of embodiments to which the principles andfeatures discussed above can be applied, it should be apparent that thedetailed embodiments are illustrative only, and should not be taken aslimiting the scope of the invention. Rather, applicant claims all suchmodifications as may come within the scope and spirit of the followingclaims and equivalents thereof.

1-14. (canceled)
 15. A method involving the rendering of information ona viewing screen, the information including foreground and backgroundinformation, the foreground information including text characters, theinformation being rendered so that the background information is notvisible on the screen at points where text characters are presented,characterized in that: the background information has a plural-bitmachine-readable data payload encoded therein, and the method includes:changing, by a first amount, a scale at which the foreground informationis rendered on the screen; and altering, by a second amount, a scale atwhich the background information is rendered on the screen; wherein theabsolute value of the second amount is less than the absolute value ofthe first amount.
 16. The method claim 15 in which the second amount iszero.
 17. The method claim 15 in which the changing is in response touser input that directs a size of a graphical window presenting saidinformation on the screen to be changed.
 18. The method claim 15 inwhich the changing is in response to user input that directs a zoomlevel at which said information is presented on the screen to bechanged.
 19. The method claim 15 in which the changing is in response touser input, and the altering is in response to said changing.
 20. Themethod claim 15 in which the background information comprises a patternhaving said plural-bit data payload steganographically encoded therein.21-42. (canceled)
 43. A method comprising the acts: changing—by a firstamount—a scale of visual content that is rendered on an electronicdisplay for user viewing; and changing—by a second amount—a scale of asteganographic digital watermark pattern that is rendered on theelectronic display with said visual content; wherein the absolute valueof the second amount is less than the absolute value of the firstamount; the method employing a hardware processor in performing at leastone of said acts.
 44. The method of claim 43 in which the absolute valueof first amount is greater than 25%.
 45. The method of claim 43 in whichthe absolute value of the second amount is less than 15%.
 46. The methodof claim 43 in which the second amount is zero.
 47. The method of claim43 in which the changing act comprises changing the scale of the visualcontent in response to user input.
 48. A non-transitory computerreadable medium containing instructions for presentation of informationon a display screen, the instructions including a portion defining usercontent for presentation on the display screen, the instructions furtherincluding watermark-related instructions causing the display screen topresent a steganographic watermark pattern with said user content, saidwatermark-related instructions causing a scale of the presentedwatermark pattern to vary less than a scale of the presented usercontent, when the scale of the presented user content is changed. 49.The medium of claim 48 in which the watermark-related instructions causethe scale of the presented watermark pattern to vary less than the scaleof the presented user content, when the scale of the presented usercontent is changed due to zooming or re-sizing of the displayedinformation.
 50. The medium of claim 48 in which the instructionsinclude HTML instructions.
 51. The medium of claim 48 in which theinstructions include at least one web API interface function selectedfrom the list consisting of window.innerWidth, window.outerWidth,window.innerHeight, window.outerHeight, document.body.clientWidth, andwindow.screen.availWidth.
 52. The medium of claim 51 in which theinstruction includes at least two web API interface functions selectedfrom said list. 53-57. (canceled)